MICROCOPY  RESOLUTION  TEST  CHART 

NATIONAL  BURIAU  OF  SIANOAROS  1%.T  A 


AD  ADA049618 

«JDC  FILE^COEtf 


ESTIMATION  OF  THE  OPERATING  CHARACTERISTICS  OF  ITEM  RESPONSE 
CATEGORIES  I:  INTRODUCTION  TO  THE  TWO-PARAMETER  BETA  METHOD 


FUMIKO  SAMEJIMA 


Department  of  Psychology 
University  of  Tennessee 
Knoxville,  TENN.  37916 


DECEMBER,  1977 

•'D  D C 


Prepared  under  the  contract  number  N00014-77-C-360, 
NR  150-402  with  the 

Personnel  and  Training  Research  Programs 
Psychological  Sciences  Division 
Office  of  Naval  Research 

Approved  for  public  release;  distribution  unlimited. 
Reproduction  in  whole  or  in  part  is  permitted  for 
any  purpose  of  the  United  States  Government. 


Unclassified 

SecuWITY  classification  of  this  page  nWin  Of  EnUnd) 

REPORT  POCUMENTATIOH  PAGE I BEFORE°C^PLETOG^ORM 

I.  sk*»OI«T  I |2.  OOVT  ACCtSSIOM  NO.  *•  RECIPIEMT'S  CATALOG  MOMBE« 

Research  Re^rt  V?-! 

n^^aracteristlcs  of  Item  Response  Categories  I:  Technical 

d Introduction  to  the  Two-Parameter  Beta^Ite^ho^  .recaFom.iNq oaorae^SR-NUMScs  ' 


|».  AOXlSOKC) 


^ FuMkoyfeame  j Ima 


1.  contsact  on  onAnr  NUMBCn^aj 


»014-77-C-036j0 


•.  nCRFORMINO  ORGANIZATION  NAME  AND  ADDREtS 

Department  of  Psychology 
University  of  Tennessee 


10.  PROGRAM  element,  PROJECT,  TASK 
AREA  A WORK  UNIT 

PE*^61153N;  PR0J:lRir  042-04 


II.  CONTROLLING  OFFICE  NAME  AND  ADDRESS  REPORT  DATE  ^ 

Personnel  and  Training  Research  Programs  C Uf>ec4Mlh«^^77) 

Office  of  Naval  Research  (code  458)  _ ” ' 

Arlington,  VA  22217  ^QQ  ^ • 

U.  monitoring  agency  name  a ADDRESS^II  dllUrmtt  from  Ctmlnlling  0/fleaJ  IS.  SECURITY  CLASSTfol  thim  npoii. 

Unclassified 


ISa.  OECLASSIFICATION/ DOWNGRADING 
SCHEDULE 

IS.  DISTRIBUTION  STATEMENT  (ol  Oil*  RapartJ 

Approved  for  public  release;  distribution  unlimited.  Reproduction 
in  whole  or  in  part  is  permitted  for  any  purpose  of  the  United  States 
government. 

17.  DISTRIBUTION  STATEMENT  (ol  Iho  aAalracI  anlaraW  In  BlocA  30,  II  dlffnranl  horn  RaporO 


I IB.  SUPPLEMENTARY  NOTES 


•t.  KEY  WORDS  (Conllnuo  on  tororoo  aiWa  II  noooooooy  ont  IWtntliy  Ay  AlacA  ■ 

Operating  Characteristic  Estimation 
Tailored  Testing 
Latent  Trait  Theory 


SO.  ABSI  RACY  fCanllmia  an  ravaraa  al#a  If  nacaaaafF  anW  IWanfliy  Ap  AfacA  miAiiJ 

^ Following  the  Normal  Approximation  Method,  in  this  study,  the  Two- 
Parameter  Beta  Method  is  developed  and  Introduced,  as  a method  of 
estimating  the  operating  characteristics  of  a test  without  assuming 

A 

any  prior  model.  The  maximum  likelihood  estimation,  9'  , and  its 
asymptotic  property  such  that  its  conditional  distribution,  given 


DD  I jAN*n  1473  edition  of  I NOV  as  IS  OBSOLETE 
S/M  0102-  LF- 014- «A0l 


Unclassified 

tECUWiry  CLASSIPICATIOH  OF  THIS  PAGE  | 


'-lOH 


lyRWFfPW 


20  coat. 


'W/- 


tCCUNITV  classification  or  this  face  fWh«i  Olm  BnlfO 

ability  6 , Is  N(0j/I(0)"*)  , where  i(0)  Is  the  test  Information 

function,  are  fully  utilized.  Data  are  the  same  set  of  simulated  data 

calibrated  and  used  In  a previous  study  In  which  the  Normal  Approximation 

Method  was  Introduced.  They  are  of  500  hypothetical  subjects  whose 
ability  levels  are  located  at  100  equally  distanced  points  of  0 between 
-2.475  and  2.475  inclusive,  with  the  interval  length  of  0.05  and  five 
subjects  at  each  point;  their  maximum  likelihood  estimates  were  estimated 
on  35  graded  items;  and  a response  pattern  of  each  subject  for  10  binary 
Items  was  calibrated  on  the  normal  ogive  model.  The  method  of  moments  is 
adopted  in  the  present  study  to  approximate  the  probability  density 
function  of  §,  using  polynomials  of  degree  3 and  4.  The  conditional 

moments  of  0,  given  §,  are  derived  from  theory  and  computed  for  each  ^ 

As. 

0 An  approximation  is  made  for  the  conditional  distribution  of  0 , 
given  0,  by  a Beta  distribution  using  the  method  of  moments,  with  two 
a priori  set  parameters  and  the  other  two  estimated  parameters  from  the 

A 

first  two  conditional  moments  of  0,  given  0.  Five  scores  of  9 are 
calibrated  following  this  approximated  Beta  distribution,  which  are 
denoted  by  0 . The  set  of  frequency  ratios  of  9 for  the  group  of 
subjects  who  have  answered  correctly  to  each  of  the  ten  binary  items 
to  the  total  group  of  subjects  is  taken  as  the  estimated  item  character- 
istic function  of  that  binary  item.  ^ The  least  square  principle  is 
adopted  in  estimating  two  parameters  in  the  normal  ogive  model  for 
each  item. 
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ESTIMATION  OF  THE  OPERATING  CHARACTERISTICS  OF  ITEM  RESPONSE 
CATEGORIES  I:  INTRODUCTION  TO  THE  TWO-PARAMETER  BETA  METHOD 

ABSTRACT 

Following  the  Normal  Approximation  Method,  in  this 
study,  the  Two-Parameter  Beta  Method  is  developed  and  intro- 
duced, as  a method  of  estimating  the  operating  characteristics 
of  a test  item  without  assuming  any  prior  model.  The  maximum 
likelihood  estimate,  § , and  its  asymptotic  property  such 

that  its  conditional  distribution,  given  ability  6 , is 
N(6,  1(0)  ^),  where  1(0)  is  the  test  information  function, 
are  fully  utilized.  Data  are  the  same  set  of  simulated  data 
calibrated  and  used  in  a previous  study  in  which  the  Normal 
Approximation  Method  was  introduced.  They  are  of  500  hypo- 
thetical subjects  whose  ability  levels  are  located  at  100 
equally  distanced  points  of  0 between  -2.475  and  2.475  inclu- 
sive, with  the  Interval  length  of  0.05  and  five  subjects  at 
each  point;  their  maximum  likelihood  estimates  were  estimated 
on  35  graded  items;  and  a response  pattern  of  each  subject 
for  10  binary  items  was  calibrated  on  the  normal  ogive  model. 

The  method  of  moments  is  adopted  in  the  present  study  to  approx- 

A 

imate  the  probability  density  function  of  0 , using  polynomials 
of  degree  3 and  4 . The  conditional  moments  of  0 , given  Q , 
are  derived  from  theory  and  computed  for  each  0 . An  approxi- 
mation is  made  for  the  conditional  distribution  of  0 , given  0 , 
by  a Beta  distribution  using  the  method  of  moments,  with  two 
a priori  set  parameters  and  the  other  two  estimated  parameters 
from  the  first  two  conditional  moments  of  0 , given  0 . Five 
scores  of  0 are  calibrated  following  this  approximated  Beta 
distribution,  which  are  denoted  by  0 . The  set  of  frequency 
ratios  of  0 for  the  group  of  subjects  who  have  answered  correctly 
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to  each  of  the  ten  binary  items  to  the  total  group  of  subjects 
is  taken  as  the  estimated  item  characteristic  function  of  that 
binary  item.  The  least  square  principle  is  adopted  in  estimating 
two  parameters  in  the  normal  ogive  model  for  each  item. 
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I.  Introduction 


A method  of  estimating  the  operating  characteristics  ot  a 
test  item  without  assuming  any  prior  model  was  proposed  by  Samejima 
(Samejima,  1977b).  Its  features  will  be  characterized  as  follows. 

(1)  We  have  a set  of  test  items  measuring  the  uni-dimensicnal 
latent  trait,  or  ability,  whose  operating  characteristics 
are  known.  For  convenience,  hereafter  we  shall  call  it  the 
Old  Test. 

(2)  The  maximum  likelihood  estimates  of  a group  of  examinees 
have  been  obtained  on  the  above  set  of  test  items. 

(3)  To  the  same  group  of  examinees  a new  item,  or  items,  has 
been  administered. 

(4)  For  the  range  of  latent  trait  in  which  all  the  examinees 
are  located,  the  test  information  function  of  the  Old  Test 
is  large  enough  that  we  can  approximate  the  conditional 
distribution  of  the  maximum  likelihood  estimate,  9 , given  latent 
trait  0,by  N(9,  1(0)^),  where  1(0)  Indicates  the 

test  information  function.  Furthermore,  the  values  of  1(9) 

are  approximately  constant  for  the  range  of  9 and,  therefore, 

-1  2 
1(0)  is  replaced  by  a 

(5)  The  bivariate  normal  distribution  is  used  as  the  approximation 

/s 

for  the  distribution  of  estimate  0 and  the  error  C , whlcl 
is  the  discrepancy  of  the  estimate  from  its  true  value  of  0 
for  each  subgroup  of  examinees  who  share  the  same  item  score 


of  a new  item. 


(6)  The  regression  coefficients  of  e on  0 , and  the 
conditional  variance  of  e , given  0 , are  computed  for 
each  Item  score  group  of  examinees  under  the  assumption  given 
In  (5),  and  five  error  scores  are  calibrated  for  each  0 by 
means  of  the  Monte  Carlo  method. 

(7)  These  error  scores  are  subtracted  from  the  respective 
maximum  likelihood  estimates,  and  the  resulting  scores  are 
denoted  by  0 . The  frequency  distribution  of  0 for  each 
score  group  Is  obtained,  using  small  equal-width  Intervals, 
and  each  frequency  Is  divided  by  the  sum  of  the  frequencies 
of  all  the  Item  score  groups  for  that  Interval. 

In  the  present  paper,  another  method  will  be  proposed 
In  a similar  setting.  The  difference  lies  In  (5)  and  (6),  and. 

Instead  of  using  the  bivariate  normal  approximation  for  each  score 
group,  we  consider  the  conditional  distribution  of  e , given  0 , 
and  approximate  It  by  a Beta  distribution.  For  convenience,  hereafter 
the  previous  method  will  be  called  the  Normal  Approxlmat Ion  Method  and 
the  present  method  will  be  called  the  2-Parameter  Beta  Method. 
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II.  Conditional  Momenta  of  Error,  Given  the  Estimate 

Let  X be  any  estimator  of  latent  trait  0 , and  n 
be  the  error  of  estimation  such  that 

(2.1)  X - e + n . 

We  assume  that  the  conditional  distribution  of  ri  , given  6 , 

2 

is  the  normal  distribution,  N(0,a  ) . It  should  be  noted  that  the 
above  conditions  are  sufficient  for  the  statistical  Independence  of  0 
and  n t obtain 

(2.2)  E(X)  - E(0)  , 

regardless  of  the  distribution  of  0 , provided  that  E(0)  exists. 

Let  ip(X|0)  denote  the  conditional  probability  density  function  of  X 

2 

given  0 , which  is  the  normal  density  function,  n(0,a  ).  Thus 
the  first  through  fourth  derivatives  of  i(j(X|0)  with  respect  to  X 
can  be  written  as  follows 


(2.3) 

^ iKX|0)  - HP(X|0) 

(2.4) 

■^2  '^(Xle)  - '^(Xl0)  a~^ 

(2.5) 

'l^(X|0)  - 3t|^(Xl0)  o"^ 

s'* 

(2.6) 

^ t|»(x|e)  - 3p(xle)  a 

From  these  results,  following  the  respective  integration  procedures, 
we  obtain  for  the  first  through  fourth  conditional  moments  of  n 
about  the  origin,  given  X , 
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(2.7) 

(2.8) 

(2.9) 

(2.10) 


li(nlA)  = g(A)][g(X)]"^  , 


A,  d 


E(n"lA)  = o"[-^2g(A)]lg(A)]  " + 0 , 


E(n’|X)  = 3a^  E(nlX)  - o^[^\g(X)][g(X)]"^  , 


E(n-lX)  = 60^  E(n^|X)  + CJ®[^  g(X)][g(X)]"^  - 


30^  , 


where  g(X)  is  the  probability  density  function  of  X , which  is 
given  by 


(2.11)  g(X)  = f(0)  .<;(X|0)  d0  . 


and  is  assumed  to  be  four  times  differentiable. 


Note  that  the  right  hand  sides  of  (2.7)  through  (2.10)  solely 
2 


consist  of  a , g(X)  and  its  derivatives.  Thus  these  conditional 
moments  are  observable,  if  we  can  fit  an  appropriate  function  for 
g(X)  based  on  our  raw  data,  or  frequency  distribution,  of  the 
estimate  X . Once  these  moments  about  the  origin  have  been  computed, 
it  is  easy  to  compute  the  corresponding  moments  about  any  arbitrary 
values  (e.g.,  Elderton  and  Johnson,  1969,  page  17). 


i ! 
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III.  Method  of  Moments  to  Graduate  tua  Set  of  Observations 

The  method  of  moments  (Elderton  and  Johnson,  1969)  has  been 
developed  for  graduating  the  observed  frequency  distribution 
using  the  observed  moments  of  up  to  a certain  degree,  assuming  a 
specified  function.  As  the  result,  we  obtain  a probability  density 
function  which  has  the  same  values  of  these  moments  and  the  specified 
functional  formula. 

If,  for  example,  we  assume  Pearson's  system  of  frequency 
curves,  we  need  the  first  four  observed  moments.  From  these  moments, 
we  can  find  out  the  value  of  the  criterion  < , defined  by 

(3.1)  K - 6^(62  + 3)^[4(232  - 33^  - 6)(432  - 33^)]"^  . 

where  3^  and  32  are  given  in  terms  of  the  second  through  fourth 
moments  about  the  mean  of  the  frequency  distribution, 
and  , such  that 

(3.2)  3j  = yj  y"^ 
and 

(3.3)  32  ■ ^2^  . 

If,  for  instance,  < turned  out  to  be  negatlvefand  finite,  then  the 
distribution  will  be  of  Pearson's  Type  I ; if  it  is  positive  and 
less  than  unity,  then  the  distribution  will  be  of  Pearson's  Type  IV  ; 
and  so  on. 

It  is  warned  by  Elderton  and  Johnson  that  we  should  avoid 


using  high  moments,  for  the  higher  the  moment  the  more  liable  Is  It 
to  error.  Although  this  is  a legitimate  warning,  this  conside rat  ion 
will  force  us  to  adopt  a relatively  simple  type  of  distribution, 
which  does  not  allow  varieties  of  possible  curves.  To  give  an  example, 
Pearson's  system  only  requires  the  first  four  moments,  but  its  curves 
for  the  probability  density  function  are,  at  most,  uni-modal,  except 
for  the  ones  which  have  U shapes.  We  must,  therefore,  balance  the  two 
opposing  factors,  i.e.,  to  avoid  the  error  and  to  allow  the  variety 
in  shape,  and  search  for  a happy  medium. 

One  solution  for  this  problem  is  to  use  a polynomial  for  the 
probability  density  function.  If  we  use  the  first  three  moments,  for 
Instance,  then  we  will  obtain  a polynomial  of  degree  3 , which  is 
expressed  as 

(3.4)  g(X)  - a + ex  + + 6X^  . 


These  four  coefficients  are  given  by  the  four  constants,  a , b , 
c and  d , such  that 


[1.125y*/R]  - [1.875y*/R^] 
[9.375y*/R^]  - [13.125y*/R^] 
[-1.875y*/R^]  + [5.625y*/R^] 
[-13.125y*/R^]  + [21.875y*/R^]  , 


through 
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fa  = a - bM 


+ cM^  - dM^ 


(3.6) 


e * b - 2cM  + 


Y = c - 3dM 


.6  = d , 


where  u*  is  the  r-th  moment  about  the  midpoint  M of  the  range 
r 

of  X , whose  length  is  2R  . If  we  add  the  fourth  moment,  then 
we  will  obtain  a pol3momial  of  degree  4 , with  the  additional  term, 
vX^  on  the  right  hand  side  of  (3.4).  In  this  case,  the  five 
coefficients  are  determined  by  the  five  constants,  a , b , c , 
d and  e , such  that 

'a-  [l.7578125y*/R]  - t8.203125y*/R^]  + [7.3828125y*/R^] 
b=  [9.375y*/R^]  - [13.125y*/R^] 

(3.7)  J c = t-8.203125y*/R^]  + [68.90625y*/R^]  - [73.828125y*/R^l 
d=  [-13.125y*/R^]  + [21.875y*/R^] 

e»  [7.3828125y*/R^]  - [73.828125y*/R^]  + [86.1328125y*/R^] 


through 


'a  * a - bM 


+ cM^  - dM^  + eM^ 


8 - b - 2cM  + 


3dM^  - 4eM^ 


(3.8)  < Y - c - 3dM  + 


6 - d - 4eM 


V 


e • 


I 
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An  advantage  of  using  a polynomial  is  that  it  allows  more  varieties 
of  curves  than  other  functions  obtainable  by  using  the  same  number 
of  moments.  It  has  a disadvantage,  however,  in  that  for  some  subsets 
of  the  variable  we  may  obtain  negative  values  for  the  probability 
density  function,  since  a polynomial  is  not  a type  of  function  defined 
for  the  probability  density. 

The  method  of  moments  described  above  can  also  be  applied 

for  a set  of  observations.  Instead  of  its  frequency  distribution. 

In  fact,  in  this  case,  we  can  preserve  more  detailed  Information 

which  may  be  lost  through  the  process  of  categorizing  the  observations 

into  the  frequency  distribution.  Neither  do  we  have  to  adjust  the 

values  of  moments  by  Sheppard's  correction,  and  so  on.  It  should  also 

be  noted  that  the  method  is  applicable  even  when  no  set  of  observations 

is  available,  but  the  estimated  values  of  moments  are  given. 

This  method  is  readily  adoptable  in  estimating  g(X)  , the 

probability  density  function  of  X , which  is  essential  in  obtaining 

the  conditional  moments  of  error  h , given  X , as  the  equations  (2.7) 

2 

through  (2.10)  show.  Besides,  together  with  o , we  can  also  use  the 
method  for  specifying  the  conditional  distribution  of  n , given  X , 
from  these  conditional  moments. 

If  we  assume  a Beta  distribution  for  the  conditional  distribution 
of  n , given  X , whose  probability  density  function,  C(n|X),,  is 
given  by 

(3.9)  C(n|A>  - [B(p,  q)]"^  (n  - a)P“^b  - n)'’"^b  - , 
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the  four  parameters,  a , b , p and  q , can  be  estimated  from 
the  four  conditional  moments  of  n , given  X . When  the  two 
parameters,  a and  b , are  known,  then  the  estimation  is  much  simpler, 
using  only  the  first  two  conditional  moments.  In  this  case,  we  have 

(3.10)  p = (1  - Mj) 

and 

(3.11)  q - (1  - - (1  - Mp  , 

where  and  are  given  by 

(3.12)  - {E(nlX)  - a}  (b  - a)"^ 
and 

(3.13)  M2  = E({n  - E(ti1X)}V)  (b  - a)"^  , 
respectively. 


I 


IV.  Maximum  Likelihood  Estimate  and  Normal  Approximation 

When  the  number  of  test  items  is  substantially  large  and  its 
test  information  is  high  enough  for  the  range  of  6 that  we  are 
Interested  in,  the  conditional  distribution  of  the  maximum  likelihood 
estimate  § , given  0 , is  approximately  N(9,  1(9)  ^ , where  1(0) 
is  the  test  Information  function  (Samejima,  1975,  1977b).  To  accept 
this  approximation,  however,  we  should  be  careful  enough  to  check  its 
fit  in  one  way  or  another.  A relatively  simple  Monte  Carlo  method 
will  take  care  of  this  for  an  actual  test  as  well  as  for  a hypothetical 
test  (Samejima,  1977b).  When  the  test  information  function  assumes  a 

constant  value  throughout  the  range  of  9 of  our  Interest,  the  variance 

-1  2 
1(0)  becomes  constant,  and  can  be  denoted  by  0 . Thus  we  can 

A 

replace  n by  e and  X by  0 in  equations  (2.8)  through  (2.11), 
to  obtain  the  conditional  moments  of  e about  the  origin,  given  0 

In  this  setting,  therefore,  we  can  adopt  the  method  of  moments 
both  for  approximating  the  probability  density  function  of  ^ and 
for  approximating  the  conditional  distribution  of  e , given  0 . 

The  greatest  merit  of  this  approach  will  be  that  we  can  do  so  without 
any  assumption  for  f(0)  , i.e.,  the  probability  density  function 
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V.  Data 

The  data  used  In  the  present  study  are  the  same  simulated 

data  that  were  used  in  the  previous  study  (Samejima,  1977b).  The 

group  of  examinees  consists  of  500  hypothetical  subjects,  whose  ability 

levels  are  located  at  100  equally  distanced  points  between  -2.475 

and  2.475  Inclusive,  with  the  Interval  length  of  0.05  and  flv.i 

subjects  at  each  point.  The  Old  Test  Is  the  set  of  35  graded  response 

Items  with,  uniformly,  three  Item  score  categories,  whose  test 

Information  function  assumes  approximately  21.63  for  the  range 

of  6 , -3.0  through  3.0  . Ten  binary  test  Items  are  used  as  new 

Items,  whose  Item  characteristic  functions  are  to  be  estimated.  The 

model  used  here  Is  the  normal  ogive  model,  for  both  the  Old  Test 

and  the  new  Items.  Each  of  the  500  subjects  has  two  response  patterns, 

l.e.,  one  on  the  Old  Test  and  the  other  on  the  set  of  the  10  binary 

Items.  The  two  parameters  of  these  ten  binary  Items,  a and  b , 

g g 

are  shown  In  Tables  1 and  2 In  a later  section. 

In  the  previous  study,  (1)  the  response  pattern  of  each  of  the!  500 
examinees  was  calibrated  by  the  Monte  Carlo  method,  (2)  the  maximum 
likelihood  estimate  was  obtained  for  each  subject  on  the  Old  Test, 
and  (3)  the  response  pattern  of  the  set  of  new  binary  Items  was 
calibrated.  Here  we  use  the  same  data. 


VI.  Method 

Following  the  rationale  described  In  previous  sections,  the 

A 

probability  density  function,  gv^.  . la  estimated  by  the  method  of 
moments  based  on  the  raw  data,  l.e.,  the  500  maximum  likelihood 
estimates.  In  so  doing,  at  most,  the  first  four  moments  are  used, 
assuming:  1)  the  Pearson's  system,  2)  a polynomial  of  degree  3 , 
and  3)  a pol}momlal  of  degree  4.  Then  the  conditional  moments  of  e 
about  the  origin,  given  § , are  computed  by  means  of  equations  (2.7) 
through  (2.10). 

The  conditional  moments  of  e about  the  origin,  given  6 , 
are  computed  using  equations  (2.7)  through  (2.l0),  and  the  first 
two  moments  are  used  to  specify  the  conditional  distribution  of  e , 

A 

given  6 . For  each  conditional  distribution,  a Beta  distribution, 
whose  probability  density  function  Is  given  In  section  3,  is  assumed, 
and  the  two  parameters,  a and  b , are  more  of  less  arbitrarily 
assigned.  From  these  two  values  and  the  first  two  moments,  the 
other  two  parameters,  p and  q , are  computed  through  (3.10)  and 
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The  simple  unweighted  least  square  method  Is  used  to  ovnluiite 
the  result,  by  obtaining  the  estimates  of  the  two  parameters  In  the 
normal  ogive  model,  as  was  used  previously  (Samejlma,  1977b).  The 


model  specifies  the  Item  characteristic  function,  Item 


g In  the  form 


(6.1) 


1_  ;t 


e du  , 


where  a Is  the  discrimination  parameter  and  b Is  the  difficulty 

s s 


parameter.  Let  P (0.)  be  the  estimated  value  of  the  Item 
S J 


characteristic  function  for  the  midpoint  of  the  J-th  Interval,  and 

2 


define  r . such  that 


_ 


e du  . 


(6.2)  P^(0)  - 

Thus  following  the  least  square  principle,  we  define  Q by 


(6.3)  2Q  - E (C  , - a„(e,  - b ))^  , 


j-1 


*gJ  g j g' 


and  differentiating  Q with  respect  to  a and  b and  setting 

g g 


the  results  equal  to  zero,  we  obtain 


(6.4) 

11. 

3a 

g 

m 

- E 

j-1 

‘^8)  - *8'®)  - ‘■s”'-®)  * ”8'  ■ ^ 

and 

(6.5) 

ia_ 

3b 

g 

m 

- E 

J-1 

<^8J  - - '>8»  *8  ■ “ • 

The  above  equations  lead  us  to  the  estimates  of  a and  b such  that 

g g 


(6.6)  a^  - Cov.  (Cgj,  ej)(Var.  (0^)) 


-1 


I 


I*. 


r 


1 


VII.  Results 

The  resulting  frequency  distribution  of  the  500  maximum 
likelihood  estimates  is  shown  in  Figures  7-1  through  7-3  , using  0.25 
as  the  interval  length.  The  first  moment  about  the  origin,  or  the 

/s 

mean,  of  the  maximum  likelihood  estimate,  9 , turned  out  to  be  -0.00577  , 

and  the  second  through  fourth  moments  about  the  mean  are  2.14824  , 

-0.01465  and  8.65145  respectively.  The  value  of  the  criterion  ^ , 
which  is  obtained  by  (3.1),  is  -0,00000762  , i.e.,  approximately 
zero.  For  this  reason,  the  distribution  In  Pearson's  system  should 
be  of  Type  II,  i.e.,  a special  case  of  the  Beta  distribution,  whose 
probability  density  function  is  given  by 

(7.1)  (B(p,  p)l“^  (§  - a)P"^  (b  - e)P~^  (b  - a}^P‘^  , 


which  is  symmetric  and  whose  mode  and  mean  are  both  (a  + b)/2  . 
The  three  parameter  values,  p , a and  b , are  estimated  from 
the  mean  and  the  variance  of  6 and  its  fourth  moment  (Elderton 
and  Johnson,  1969;  Johnson  and  Kotz,  1970)  in  such  a way  that 


(7.2)  $ - 3(32  " ■ ^2^  ‘ 

(7.3)  a - £(§)  - (2  Var.  (§)  (3  - 82)"^^^ 


and 


(7.4)  B - E(e)  + (2  Var.  (0)  (3  - 82)'^'^^  , 


where  82  Is  defined  by  (3.3)  . These  values  turned  out  to  be  1.16581  , 
-2.68111  and  2.66947  respectively.  Figure  7-1  presents  the 
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probability  density  function  of  the  Pearson's  Type  II  distribution 
using  the  above  estimated  parameters,  together  with  the  frequency 
distribution  of  § mentioned  earlier. 


The  result  of  fitting  a polynomial  for  the  probability  density 
function  of  0 gave  the  estimated  coefficients: 


0.22416 

-0.00351 

-0.01873 

0.00095 


for  degree  3 , and  for  degree  4 we  have 


0.19620 
0.00238 
0.01319 
-0.00062 
-0.00427  . 


Figures  7-2  and  7-3  present  these  two  estimated  probability  density 
functions  in  a similar  manner  as  Figure  7-1.  These  three  results 
will  be  more  clearly  Illustrated  if  we  draw  the  curves  of  their 
distribution  functions,  and  those  of  the  cumulative  frequency 
distribution  of  the  maximum  likelihood  estimate  6 . Figure  7-4 
through  7-6  present  these  results,  in  the  same  order  as  Figure  7-1 
through  7-3. 

It  is  clear  from  these  figures  that  the  discrepancy 
between  the  frequency  distribution  and  the  theoretical  density 


FIGURE  7-2 

The  frequency  distribution  of  the  maximum  likelihood  estimate  (histogram) , its 
graduated  polynomial  of  degree  3 (solid  curve)  and  the  theoretical  density  of 
the  maximum  likelihood  estimate  (dotted  curve) . 
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FIGURE  7-4 

The  distribution  function  obtained  from  the  graduated  Pearson's  Type 
II  function  (dashed  curve)  and  the  cumulative  frequency  distribution 
of  the  maximum  likelihood  estimate  (solid  curve). 
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The  distribution  function  obtained  from  the  graduated  polynomial  of 
degree  3 (dashed  curve)  and  the  cumulative  frequency  distribution  of 
the  maximum  likelihood  estimate  (solid  curve). 
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FIGURE  7-6 

distribution  function  obtained  from  the  graduated  polynomial  of 
ee  4 (dashed  curve)  and  the  cumulative  frequency  distribution  of 
the  maximum  likelihood  estimate  (solid  curve). 
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is  relatively  large.  This  fact  is,  of  course,  due  to  the 
sampling  fluctuation  in  the  process  of  item  score  calibration 
for  each  item  of  the  Old  Test,  on  which  the  maximum  likelihood 
estimate  was  derived.  Since  each  function  was  graduated 
on  our  raw  data,  there  is  some  conspicuous  difference  between 
the  graduated  curve  and  the  curve  for  the  theoretical  density, 
especially  in  Degree  3 Case. 


In  evalutalng  these  three  outcomes,  it  will  be  more  helpful 
if  we  introduce  the  theoretical  probability  density  function,  g(6)  . 
This  is,  in  general,  given  by  (2.11),  replacing  X by  6 . With 
our  data,  it  is  appropriate  to  assume  a uniform  distribution  for 


the  distribution  of  0 , and  we  can  write 


(7.7)  g(0)  * c (2tt) 


expf-(0  - 0)^/2a^}  d0 


^ V 

(2tt)“^^^  I expf-(0  - 0)^/2a 

J 0 

-1/9  9 

(271)  I ^ exp{-t  /2}  dt  , 

J (0-0) /a 


defining 


1 (7.8)  t - (9  - 0)a"^  , 

'i 

j 

where  c is  the  constant  probability  density  of  9 for  the  closed 
interval  [Q_,  . With  our  data,  c - 0.2  , -2.5  and  ?-  2.5  , and 

o “ 0.215  , which  is  the  square  root  of  the  inverse  of  the  constant 
test  information  function,  ,21.63  . The  tightest  hand  side  of  (7.7) 
is  shown  in  Figures  7-1  through  7-3  by  dotted  curves. 


Comparing  the  two  polynomials,  it  is  obvious  that  the  polynomial 


of  degree  4 provides  us  with  the  closer  curves  to  both  the  frequency 
distribution  of  0 and  its  theoretical  probability  density  function, 

, than  the  polnomial  of  degree  3.  This  fact  is  also  observed  in 

the  comparison  of  the  two  distribution  functions  in  Figures  7-5 
' and  7-6.  This  is  an  expected  result,  and  there  is  no  doubt  that 

we  should  choose  the  pol3momlal  of  degree  4 , if  the  choice  should 
be  made  between  these  two. 

As  for  the  Pearson's  Type  II  distribution,  it  should  be  noted 
that,  at  least,  the  symmetry  of  the  theoretical  probability  density 
function  is  realized.  Because  of  the  restriction  coming  from  the 
nature  of  the  Beta  distribution,  however,  the  curve  in  Figure  7-1 
is  conspicuously  different  from  the  theoretical  probability  density 
function  at  both  extreme  values  of  0 . If  we  compare  this  with  the 
curve  of  the  polynomial  of  degree  4 given  in  Figure  7-3,  we  will 
find  out  this  problem  is  substantially  ameliorated  in  the  latter. 

Also  the  comparison  of  Figures  7-4  and  7-6  convinces  us  that  the  fit 
of  the  polynomial  of  degree  4 is  better  than  that  of  the  Pearson's 
Type  II  function,  in  spite  of  the  fact  that  both  functions  require 
the  first  through  fourth  moments  in  the  parameter  estimation. 

From  all  these  observations,  it  will  be  concluded  that  the 
polynomial  of  degree  4 provides  us  with  the  best  fitted  curve  among 
the  three,  and  that  of  degree  3 gives  us  the  worst  fitted  one.  A 
close  observation  of  Figure  7-6  tells  that  a further  Increase  in 
I degree  may  not  be  necessary, considering  that  we  must  use  the  fifth 

I moment  of  0 to  do  that.  (This  will  be  confirmed  by  a later  observation.) 
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For  this  reason,  hereafter  we  use  the  polynomial  of  degree  4 as  the  I 

estimated  probability  density  function  for  the  maximum  likelihood 
estimate.  We  shall  also  use  the  polynomial  of  degree  3 in  addition, 
to  observe  how  much  accuracy  is  lost  by  using  a relatively  poorly 

fitted  function.  For  convenience,  hereafter  we  shall  call  these  i 

two  cases  Degree  4 Case  and  Degree  3 Case  respectively. 


Using  the  estimated  g(&)  in  Degree  4 Case,  the  first  through 
fourth  conditional  moments  of  the  error  e , given  ^ , were  computed 
for  each  value  of  the  500  maximum  likelihood  estimates  through 
the  formulas  (2.8)  through  (2.11).  As  is  expected  from  Figure  7-3, 
however,  it  was  impossible  to  do  so  for  one  value  of  the  maximum 
likelihood  estimate,  -3.0555  , since  the  estimated  probability 
density  assumes  a negative  value.  This  looks  like  a deficiency  in 
using  a polynomial  for  the  probability  density  function,  since  it 
is  not  originally  developed  as  one.  A close  examination  of  Figure  7-1, 
however,  will  tell  us  that,  if  we  use  the  Pearson's  Type  II  distribution 
instead,  we  will  have  more  than  one  value  of  the  maximum  likelihood 
estimate  for  which  the  estimated  probability  density  function  assumes 
negative  values,  although  the  function  itself  is  a probability  density 
function.  In  fact,  if  we  adopt  the  Pearson's  Type  II  distribution,  then 
we  will  have  ten  values  of  the  maximum  likelihood  estimates  which  are 
outside  of  the  interval,  (-2.68111,  2.66947)  , and,  therefore, 
the  estimated  g(@)  assumes  zero,  a result  that  is  worse  than  that 
of  Degree  4 Case. 

In  addition  to  the  above  problem,  in  Degree  4 Case,  several 


i 


negative  values  are  observed  for  even  conditional  moments.  For  two  | 

A. 

cases  where  0 Is  no  greater  than  -2.7417  and  for  four  cases 
where  It  is  no  less  than  2.7137  , the  second  conditional  moments 
about  the  mean  turned  out  to  be  negative.  In  addition  to  these 

A 

six  cases,  for  four  cases  where  0 is  no  greater  than  -2.6723  and 

for  three  cases  where  it  is  no  less  that  2.6346  the  fourth  conditional  ^ 

J 

moment  about  the  mean  turned  out  to  be  negative.  We  must  exclude  1 

seven  cases,  therefore,  if  we  use  up  to  the  second  moments,  and 
fourteen  cases  if  we  use  up  to  the  fourth  moments,  in  Degree  4 Case. 

A 

On  the  other  hand,  there  is  only  one  case,  l.e.,  0 ■ -3.0555  , in 

which  the  fourth  conditional  moments  turned  out  to  be  negative,  and 
all  the  500  second  conditional  moments  are  positive,  in  Degree  3 Case. 

It  will  be  of  theoretical  propriety  that  we  use  the  four 
conditional  moments  to  graduate  the  conditional  distribution  of  e , 
given  § , assuming  some  appropriate  functional  formula.  The 
above  observation  for  Degree  4 Case  makes  us  wonder,  however,  if  it 
Is  worth  trying,  taking  the  risk  of  using  higher  moments  which  are 
liable  to  error.  For  this  reason.  In  the  present  study,  only  the 
first  two  conditional  moments  for  each  maximum  likelihood  estimate 
were  used.  The  seven  cases.  In  which  the  second  moments  are  negative. 


were  excluded  In  Degree  4 Case,  therefore,  and  the  remaining  493 
cases  were  used,  whereas  In  Degree  3 Case  all  the  500  cases  were  used. 
The  selection  of  an  appropriate  functional  formula  for  the 

A 

conditional  probability  density  function  of  e , given  0 , Is  rather 


difficult.  If  we  use  only  the  first  two  moments.  It  Is  desirable  that 
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the  function  allows  a variety  of  different  curves,  including  simple 

ones  as  well  as  complicated  ones,  to  simulate  the  unobserved  probability 

functions.  If,  for  instance,  we  use  the  normal  density  function, 

then  the  curve  is  always  uni-modal  and  symmetric,  which  is  not  desirable. 

The  polynomial  of  degree  2 also  has  the  same  problem,  in  addition 

to  the  possibility  of  producing  negative  values  for  some  subset  of 

the  domain.  In  any  case,  it  is  difficult  to  avoid  the  uni-modality 

of  the  curves.  The  symmetry  of  the  curves  can  be  avoided,  however, 

if  we  use  the  Beta  density  function,  or  Pearson's  Types  1 and  II,  assuming 

appropriate  sets  of  two  parameters,  i.e.,  a and  b in  (3.9).  It 

is  to  our  benefit  that  the  Beta  distribution  includes  a variety  of 

different  curves  for  its  density  function  (e.g.,  Johnson  and  Kotz,  1970, 

page  44),  such  as  straight  lines,  J-shape  curves,  U-shape  curves, 

symmetric  and  non-symmetrlc  uni-modal  curves,  etc.  For  this  reason, 

in  the  present  study,  the  Beta  distribution  was  adopted  for  the  conditional 

A. 

distribution  of  e , given  0 , with  the  assumed  values  of  two  parameters, 
a and  b , for  each  maximum  likelihood  estimate.  The  question  is  how 
to  determine  the  values  of  a and  b > arid  it  was  answered  more  or  less 
arbitrarily.  Following  the  logic  of  interval  estimation  when  the 
probability  density  function  of  6 is  not  known,  for  each  value  of  the 
maximum  likelihood  estimate  the  value  of  0 , for  which  the  probability 
of  obtaining  that  value  of  the  maximum  likelihood  estimate,  or  greater, 
is  approximately  0.0054  , is  specified,  and  used  as  a , and  the 
corresponding  value  of  6 , for  which  the  probability  of  obtaining 
that  value  of  the  maximum  likelihood  estimate,  or  less,  is  approximately 
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0.0054  , was  used  as  b . Thus  we  have 


(7.9) 


ag  = § - (0.215  X 2.55)  or  a-  = 0 - 0.54825 


I'ag  = 0 - (0.215  X 2. 
» 0 + (0.215  X 2. 


55)  or  bg  - 0 + 0.54825  . 


The  other  two  parameters,  pj;  and  qg  , were  estimated  from  the 

0 0 

first  two  moments  and  ag  and  bg  , through  (3.10)  and  (3.11). 

0 0 

Following  the  previous  study  (Samejlma,  1977b),  five  scores, 

0 , are  calibrated  for  each  of  the  500  maximum  likelihood  estimates, 
according  to  the  Beta  distribution  thus  specified.  As  the 
result,  we  obtained  2500  0 in  Degree  3 Case,  and  2465  0 in  Degree  4 

Case.  Figures  7-7  and  7-8  present  the  frequency  distributions  of  0 
in  Degree  3 and  4 Cases  respectively,  together  with  the  distributions 
of  9 . As  was  mentioned  earlier,  the  five  hundred  values  of  the 
latent  trait  0 are  placed  between  -2.475  and  2.475  inclusive, 
forming  subgroups  of  five  each  at  the  interval  length  of  0.05,  so  as 
the  frequency  distribution  the  rectangle  shown  in  Figures  7-7  for 
Degree  3 Case  is  the  case.  In  Degree  4 Case,  however,  we  had  to  exclude 
Subject  2 with  0 “ -2.425  because  of  the  negative  estimated  probability 
function  of  0 , and  Subjects  99,  101,  201,  296,  299  and  300  because  of 
their  negative  values  of  estimated  second  conditional  moments 
about  the  mean,  whose  0 are  2.425  , -2.475  , -2.475  , 2.275  , 

2.425  and  2.475  respectively.  The  histogram  shown  in  Figure 
7-8  differs  slightly  from  the  rectangle,  therefore,  at  both  ends 
of  the  interval  of  0 . The  estimated  conditional  moments  for 
each  of  the  500  maximum  likelihood  estimates,  together  with  the 
values  of  8^^  , 3^  criterion  k , are  given  in  Appendix 

I,  as  Table  A-1-1  for  Degree  3 Case  and  Table  A-1-2  for  Degree  4 
Case  respectively. 


200 


The  frequency  distributions  of  2500  6 in  Degree  3 Case  (solid  line)  and  of  ability 
6 (dotted  line).  The  latter  is  multiplied  by  five  for  comparison. 
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The  frequency  distributions  of  2465  0 in  Degree  4 Case  (solid  line)  and  of  ability 
9 (dotted  line).  The  latter  is  multiplied  by  five  for  comparison. 
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Comparing  these  two  results,  we  can  see  that  the  fit  of  the 
frequency  distribution  of  0 to  that  of  0 is  slightly  better  for 
Degree  4 Case  than  for  Degree  3 Case,  especially  at  both  ends  of 
the  Interval  of  6 . If  we  compare  Figure  7-7  with  the  five  histograms 
of  0 obtained  for  Items  2,  4,  6,  8 and  10  by  the  Normal  Approximation 
Method,  however,  we  can  see  that  the  fit  for  Degree  3 Case  is  as 
good  as  those  in  these  five  cases  (cf.  Samejima,  1977b,  Figure  7, 
page  187). 

For  each  of  the  ten  binary  items,  the  500  examinees  are 
classified  into  two  groups,  i.e.,  the  group  of  those  who  answered 
item  g correctly,  and  that  of  those  who  responded  incorrectly. 

This  procedure  also  divides  the  2500  or  2465  0 into  two  groups, 
with  respect  to  each  item.  Thus  the  frequency  distributions  shown 


J • 


In  Figures  7-7  and  7-8  are  divided  into  two  smaller  frequency 
distributions  respectively,  and,  for  each  Interval,  the  frequency 
ratio  of  the  frequency  for  the  "correct"  group  to  that  for  the 
total  group  provide  us  with  the  estimated  value  of  the  item 
Characteristic  function  of  the  item.  Figure  7-9  presents  these  frequency 
ratios,  together  with  the  corresponding  results  obtained  by  the  Normal 
Approximation  Method  (Samejima,  1977b),  for  each  of  the  ten  binary  items. 
In  these  eleven  graphs,  the  results  obtained  on  500  6 in  the  previous 

study  are  plotted  with  small  hollow  circles,  those  on  2500  0 and  on 
5000  0 are  with  large  circles  and  small  circles,  each  of  which  is 
put  in  a large  hollow  circle,  respectively,  and  those  obtained  in  the 
present  study  for  Degree  3 and  4 Cases  are  with  triangles  and  squares 
respectively.  Except  for  the  first  group,  the  plots  are  solid  in  the 
range  of  (0.05,  0.95),  and  hollow  otherwise. 
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Item 


Item 


Addition:  the  result  obtained  by  using  2500  6 in  the  Normal  Approximation  Method 

(large  circle) . 
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FIGURE  7-9 
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LATENT  TRAIT  0 

FIGURE  7-9 
(Continued) 

Addition:  the  same  as  Items  2 and  4. 
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(Continued) 

Addition:  the  same  as  Items  2,  4 and  6(A). 
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Addition:  the  sane  as  Items  2,  4,  6(A)  and 
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The  numbers  of  hypothetical  subjects  who  belong  to  the 
"correct"  and  "incorrect"  groups  for  each  of  the  ten  binary 
items  are  given  in  Appendix  II  as  Table  A-2-1  . Each  frequency 
in  this  table  multiplied  by  5 makes  the  frequency  of  0 for 
that  category,  for  Degree  3 Case.  In  Degree  4 Case,  each 
frequency  added  to  the  negative  number  shown  in  the  brackets 
and  then  multiplied  by  5 makes  the  frequency  of  0 for  that 
category . 

The  estimates  of  the  two  parameters  of  the  normal  ogive  model, 

S and  6 , were  obtained  through  the  method  described  in  section  6. 

g g 

These  values  are  shown  in  Tables  7-1  and  7-2,  together  with  the  true 
parameter  values.  In  adopting  the  above  method,  those  frequency 
ratios  which  are  less  than  0.05  or  greater  than  0.95  are 
excluded.  As  the  result,  the  numbers  of  Intervals  used  range  from 
3 to  14  , and  these  numbers  are  also  presented  in  Tables  7-1 
and  7-2.  Similar  computations  were  made  for  both  a and  ti  , 

O O 

by  changing  the  cutting  points  of  the  frequency  ratio  to  J)  0.15 
and  0.85  , 2)  0.J.0  and  0.90  , and  3)  0.01  and  0.99  . These 

results  are  presented  in  Appendix  III  as  Tables  A-3-1  and  A- 3-2, 
together  with  those  obtained  by  the  Normal  Approximation  Method. 
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VIII.  Further  Observations  and  Analyses  of  the  Results 

The  results  presented  in  the  preceding  section  indicate 

that,  in  the  present  study,  the  2-Parameter  Beta  Method  has  worked 

as  well  as  the  Normal  Approximation  Method.  There  is  no  distinct 

indication,  however,  that  the  present  method  is  better  than  the 

previous  one,  although  it  contains  more  mathematical  sophistication. 

We  can  say  that,  in  general,  the  P (0)'s  obtained  by  the  2-Parameter 

S 

Beta  Method  show  better  fits  to  the  theoretical  P (6)'s  than  do  those 

g 

obtained  by  the  Normal  Approximation  Method  using  500  6's  , but  they 

are  just  as  good  as  those  obtained  by  the  Normal  Approximation  Method 
using  2500  O's  (cf.  Figure  7-9). 


As  for  the  estimates  of  the  two  parameters,  a and  b , it 

g g 

is  observed  that,  in  general,  the  estimates  of  the  difficulty 
parameter,  b^  , are  close  enough  to  the  true  parameter  values. 

Figure  8-1  presents  these  estimates  plotted  against  the  true  parameter 
values  for  both  Degree  3 and  4 Cases.  We  can  see  that  all  these  plots 
are,  at  least,  reasonably  close  to  the  line  intercepting  the  origin  with  an 
angle  of  45  degrees  from  the  abscissa.  We  notice,  however,  that  the  estimates 
of  the  discrimination  parameters  tend  to  be  less  than  the  true  parameter 


I : • 


values.  Figure  8-2  presents  the  estimates  of  a^  for  the  ten  items 
plotted  against  the  true  parameter  values,  and  shows  this  tendency 
clearly.  The  same  has  been  observed  in  the  results  of  the  Normal 
Approximation  Method,  and  this  Inaccuracy  in  estimating  the  discrimination 
parameter  is  a subject  we  should  work  on  in  the  future.  Unlike  the 


Normal  Approximation  Method,  the  2-Parameter  Beta  Method  leaves  much 
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room  for  improvement  In  accuracy,  including  the  adjustment  of  the 
aprlorl  set  parameters,  ag  and  bg  . In  the  present  study,  they 
are  set  equal  to  (0+  2'.55a),,  and  this  may  be  too  wide  an  interval. 
Further  investigation  will  be  done  in  a later  study. 

There  is  evidence  which  debates  the  above  possibility  and  will 
explain  the  reason  for  the  inaccuracy  in  estimating  a^  , however. 
Table  8-1  presents  the  results  of  the  discrimination  parameter 
eslmlnatlon  obtained  directly  from  the  frequency  ratios  of  the  true  9 , 
by  the  same  method  using  the  twenty  equal  length  Intervals  with  the 
total  frequency  of  25  for  each  interval.  We  can  see  that  for  nine 
items  the  estimated  discrimination  parameters  are  less  than  the  true 
parameter  values,  and  these  values  are  very  close  ^ the  estimates 
obtained  in  Degree  3 and  4 Cases.  If  we  consider  this  fact,  we  must 
say  that  the  2-Parameter  Beta  Method,  as  well  as  the  Normal  Approximation 
Method,  has  worked  very  well.  In  the  same  table,  the  results  obtained 
directly  from  the  frequency  ratios  of  the  500  maximum  likelihood 
estimates  are  also  presented,  for  two  situations  where  all  the  20 


intervals  are  used  and  where  only  16  intervals  with  the  total  frequency 
greater  than,  or  equal  to,  20  are  used.  As  is  expected  from  theory 
(Samejlma,  1977b),  for  most  items  the  estimates  of  the  discrimination 
parameter  are  less  than  those  obtained  directly  from  the  frequency 
ratios  of  the  true  values  of  0 . Similar  estimates  obtained  by  changing 
the  cutting  points  of  the  frequency  ratio  to  each  of  the  three  different 
sets,  0»15  and  0.85  , 0.10  and  0.90  , and  0.01  and  0.99  , are  present- 


ed in  Appendix  IV  as  Table  A-4-1.  The  corresponding  tables  for  the 
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TABLE  8-1 

The  Discrimination  Parameter  and  Its  Estimates  of  Each  of 
the  Ten  Binary  Items  Obtained  Directly  from  the  Frequency 
Ratios  between  0.05  and  0.95  of  the  True  9 , and  from 

Those  of  the  Maximum  Likelihood  Estimates  Using  20  and  16 
Intervals  Respectively 


METHOD 


ITEM 

TRUE 

a 

g 

a from  6 
g 

0.05- 

0.95 

a f rom  MLE 
g 

(20  points) 

0.05- 

0.95 

a from  MLE 
g 

(16  points) 

0.05- 

0.95 

1 

1.5 

2.250 

0.876 

1.206 

3 

4 

3 

2 

1.0 

0.980 

1.137 

1.195 

3 

2.5 

2.369 

1.973 

2.111 

4 

1.0 

0.862 

0.794 

0.897 

5 

1.5 

1.327 

1.297 

1.222 

6 

1 .0 

0.778 

0.835 

0.915 

7 

2 .0 

1.532 

1.428 

1.382 

8 

1.0 

0.943 

0.785 

0.749 

9 

2.0 

1.889 

1.810 

1.869 

10 

1 .0 

0.727 

0.820 

0.534 

The  number  of  Intervals  used  in  estimation  is  shown  as  a subscript 
when  it  is  less  than  6 . 


TABLE  8-2 


The  Difficulty  Parameter  and  Its  Estimates  of  Each  of  the 
Ten  Binary  Items  Obtained  Directly  from  the  Frequency  Ra- 
tios between  0.05  and  0.95  of  the  True  6 , and  from  Those 
of  the  Maximum  Likelihood  Estimates  Using  20  and  16 
Intervals  Respectively 


METHOD 

TRUE 

b 

g 

S from  0 
g 

S from  MLE 
g 

(20  points) 

b f rom  MLE 
g 

(16  points) 

ITEM 

0.05- 

0.05- 

0.05- 

0.95 

0.95 

0.95 

1 

-2.5 

-2.411 

3 

-3.010 

4 

-2.631 

3 

2 

-2.0 

-2.023 

-1.953 

-1.913 

3 

-1.5 

-1.478 

-1 .495 

-1.482 

4 

-1.0 

-0.957 

-1.050 

-1.119 

5 

-0.5 

-0.442 

-0.492 

-0.445 

6 

0.0 

0.001 

-0.062 

-0.112 

7 

0.5 

0.577 

0.518 

0.580 

8 

1.0 

0.969 

0.959 

0.905 

9 

1.5 

1.514 

1.511 

1.507 

10 

2.0 

2.117 

2.012 

2.495 

The  number  of  Intervals  used  in  estimation  is  shown  as  a subscript 
when  it  is  less  than  6 . 
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difficulty  parcimeter  b are  presented  as  Table  8-2  and  Table  A-4-2 

8 

in  Appendix  IV  respectively. 

Figure  8-3  presents  these  frequency  ratios  of  9 and  those 
of  the  maxiraura  likelihood  estimate.  The  comparison  of  these  ten 
graphs  with  those  in  Figure  7-9  makes  it  obvious  that  the  frequency 
ratios  are  by  no  means  closer  to  the  theoretical  curves  than  those 
obtained  by  the  2-Parameter  Beta  Method,  or  by  the  Normal  Approximation 
Method. 

From  these  observations,  it  will  be  concluded  that  the 
2-Parameter  Beta  Method  has  proved  itself  to  be  useful  in  estimating 
the  operating  characteristics  of  item  response  categories.  It  has 
also  been  shown  that  there  is  little  evidence  to  support  Degree  4 
Case  in  preference  to  Degree  3 Case,  regardless  of  the  distinct 
difference  between  the  two  graduated  curves  (cf.  Figures  7-2  and  7-3), 
at  least,  in  the  present  study.  This  does  not  encourage  us  to  use 
a polynomial  of  a higher  degree  to  graduate  the  raw  data  of  maximum 
likelihood  estimates  in  the  present  study.  In  fact,  the  method  of 
moments  to  fit  a polynomial  of  degree  5 to  g(^)  requires  up  to  the 
fifth  moment,  and  the  coefficients  in  the  form 

(8.1)  g(X)  - a + BX  + 


are  computed  by 
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2cM  + 3dM^  - 4eM^  + 5fM^ 

3dM  + 6eM^  - lOfM^ 

4eM  + lOfM^ 

5fM 
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where 


f: 


(8.3) 


e 

. f 


[1.7578125y*/R]  - [8.203125y^/R^]  + [7.3828125y*/R^] 
[28.7109375y*/R^J  - [ 103. 359375y*/R^]  + [81.2109375y*/R^] 
[-8.203125yg/R^]  + [68.90625y5/R^l  - t73.828125yj/R^] 
[-103.359375yJ/R^]  + [442 . 96875y*/R^]  - [378.984375y*/R^] 
[7.3828125yg/R^]  - [73.828125y5/R^]  + [86. 1328125yJ/R^] 
[81.2109375yJ/F^]  - [378.984375y*/R^]  + [341.0859375y^/R^^] 


With  our  raw  data  of  the  500  maximum  likelihood  estimates,  these  values 
turned  out  to  be: 


(8.4) 


f a 

B 

Y 


0.19539 

(0.19620) 

-0.00638 

(0.00238) 

0.01449 

(0.01319) 

0.00405 

(-0.00062) 

-0.00449 

(-0.00427) 

-0.00048  . 

The  values  in  the  brackets  in  (8.4)  are  corresponding  estimates  for 
the  polynomial  of  degree  4,  which  were  Introduced  in  section  7.  We 
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notlce  that.  If  we  take  three  places  below  the  decimal  point,  these 
two  sets  of  coefficients  are  almost  identical.  Figure  8-4  shows 
this  polynomial  of  degree  5,  and  it  is,  indeed,  very  similar  to  the 
polynomial  of  degree  4,  which  was  presented  as  Figure  7-3  in  the 
preceding  section.  In  spite  of  this  result,  however,  we  should  be 
cautious  about  the  adoption  of  a relatively  simple  polynomial  in 
preference  to  polynomials  of  higher  degrees.  As  is  shown  in  Figures  7-1, 
7-2,  7-3  and  8-4,  the  theoretical  probability  function  of  the  maximum 


likelihood  estimate  in  the  present  study  has  a relatively  simple  curve. 


but  we  should  not  expect  this  is  always  the  case.  A polynomial  of  a 
higher  degree  will  be  appreciated  if  this  theoretical  function  is 


more  complicated,  like  one  having  three  modal  points,  and  so  on.  As 
was  warned  earlier,  however,  the  adoption  of  a polynomial  of  a higher 
degree  requires  us  to  use  momenta  of  higher  degrees,  which  are  liable 
to  error.  To  solve  this  problem,  it  may  be  advisable  to  divide  the  total 
group  of  subjects  into  several  subgroups,  and  fit  a polynomial  of  a 
relatively  low  degree  to  each  subset  of  the  maximum  likelihood  estimates. 
For  the  purpose  of  illustration.  Figure  8-5  presents  the  results 
obtained  by  using  five  subgroups,  each  of  which  consists  of  100  subjects, 
and  polynomials  of  degree  3 and  4 respectively.  Although  these  sets 
of  curves  are  too  complicated  to  use,  they  give  us  the  idea  that  it  is 
possible  to  graduate  our  raw  data  fairly  accurately,  without  using 
polynomials  of  higher  degrees.  The  estimated  coefficients  of  these 
five  polynomials  are 
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FIGURE  8-5 

The  five  polynomials  graduated  for  five  separate  Intervals  (dashed  curve) , th 
frequency  distribution  of  the  maximum  likelihood  estimate  (histogram)  and  the 
theoretical  density  of  the  maximum  likelihood  estimate  (dotted  curve) . 
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(8.6)  < 


for  those  of  degree  4.  The  intervals  of  the  maximum  likelihood 
estimate  used  for  the  five  subgroups  are  [-3.0555,  -1.5096]  , 

[-1.4941,  -0.5265]  , [-0.5265,  0.4771]  , [0.4852,  1.5297]  and 

[1.5395,  2.8718]  , respectively.  Figure  8-6  presents  similar  results 
obtained  by  using  two  subgroups,  each  of  which  has  250  subjects,  for 
polynomials  of  degrees  3 and  4 respectively.  The  estimated  coefficients 
are 


/ ^ 

a 

; 0.20878 

0.21750  , 

: 0.03405 

-0.11653 

(8.7)  . 

Y 

; 0.05388 

0.13965 

: 0.02235 

-0.04411 

for  the 

two 

polynomials 

of  degree  3,  and 

1 


II 


a 

-1.17587 

0.56491 

0.19259 

2.67749  2.00465 

-1.91141 

0.91174 

0.05255 

-8.27070  -2.16327 

Y 

-0.82751 

0.74415 

0.07557 

8.59277  0.88624 

8 

-0.10701 

0.20091 

-0.25630 

-2.81832  -0.13030 

ol3nnomials  of 

degree  3, 

and 

/N 

a 

15.22161 

7.81272 

0.18580 

8.70223  25.44446 

B 

28.35231 

32.93790 

0.06613 

-35.55154  -46.44038 

ys 

Y 

19.74585 

51.74863 

0.34740 

52.73598  31.81450 

8 

6.00195 

34.99721 

-0.38289 

-33.19569  -9.60191 

V 

0.66909 

8.61039 

-1.28133 

7.53819  1.07356 
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FIGURE  8-6 

The  two  polynomials  graduated  for  two  separate  intervals  (solid  curve),  the 
frequency  distribution  of  the  maximum  likelihood  estimate  (histogram)  and  the 
theoretical  density  of  the  maximum  likelihood  estimate  (dotted  curve) . 


FIGURE  8-6 
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(8.8)  ^ Y 

s 


0.20953  0,2A402 
0.03869  -0.31223 
0.06056  0.45212 
0.02572  -0.21494 
0.00055  0.02991 


for  those  of  degree  4.  The  intervals  of  the  maximum  likelihood  estimate 
for  the  two  subgroups  are  [-3.0555,  -0.0200]  and  [-0.0159,  2.8718] 
respectively.  It  is  interesting  to  note  that  the  two  sets  of  results 
are  much  more  similar  to  each  other  than  those  obtained  without  dividing 
the  total  set  of  raw  data,  and  especially  for  the  first  subgroup  the  two 
polynomials  are  very  close. 

The  value  of  the  criterion  < , which  was  introduced  in  section  3, 
was  caluculated  for  each  of  the  500  maximum  likelihood  estimates,  for 
both  Degree  3 and  4 Cases.  The  result  shows  that,  in  Degree  3 Case,  318 

A. 

conditional  distributiore  of  0 , given  the  maximum  likelihood  estimate  9 
belong  to  Pearson's  Type  I,  and  181  are  normal  distributions,  with  one 
case  for  which  the  fourth  conditional  moment  turned  out  to  be  negative 
and,  therefore,  no  distribution  is  specified  ; in  Degree  4 Case  432 
conditional  distributions  are  of  Type  I and  54  are  of  Type  II,  with  the 
exceptions  of  13  cases  for  which  the  fourth  and/or  the  second  conditional 
moments  are  negative  and  one  case  for  which  the  estimated  probability 
density  Itself  is  negative. 
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IX.  Discussion  and  Conclusion 

The  2-Parameter  Beta  Method  was  proposed  and  adopted  to  estimate 
the  item  characteristic  functions  of  ten  hypothetical  binary  items  in 
the  simulation  study,  and  the  results  are  compared  with  those  obtained 
by  the  Normal  Approximation  Method,  and  others.  It  can  be  concluded 
that  the  method  proved  to  be  useful.  On  the  other  hand,  more  intensive 
research  is  desired  to  further  improve  the  accuracy  of  the  estimation. 
Unlike  the  Normal  Approximation  Method,  it  is  very  probable  that  such 
an  effort  will  be  rewarded,  by  investigating  an  optimal  set  of  two 
fixed  parameters,  ag  and  bg  , and  so  on. 

As  was  presented  at  the  end  of  the  preceding  section,  the 
criterion  < shows  that  the  majority  of  the  conditional  distributions 
of  0 , given  the  maximum  likelihood  estimate  0 , are  of  Pearson's 
Type  I or  II  in  both  Degree  3 and  4 Cases,  although  it  is  interesting 
to  note  that  about  36  percent  of  them  in  Degree  3 Case  are  normal 
distributions.  It  is  expected,  therefore,  that  the  accuracy  of 
estimation  increases  if  we  approximate  these  conditional  distributions 
by  Beta  distributions  without  fixing  any  parameters,  though  this  will 
cost  more  numerical  elaboration.  This  will  be  attempted  in  a later 
study,  as  well  as  further  intensive  research  on  the  2-Parameter  Beta 
Method. 

It  is  also  desirable  to  use  different  sets  of  data  in  our  future 
studies.  In  particular,  in  the  present  study,  the  value  of  a is  as 
small  as  0.215  , and  it  will  be  worthwhile  to  use  other  simulation  data 
with  a larger  value  of  c . This  will  easily  be  done  by  using  simulated 


! 
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tailored  testing  data  in  which  a fixed  amount  of  test  information  Is 
used  as  the  criterion  for  terminating  the  presentation  of  new  items 
(Samejima,  1977a,  1977d),  rather  than  using  a hypothetical  paper-and-pencil 
test,  since  the  whole  procedure  in  such  a tailored  testing  situation  can 
be  considered  as  producing  a weak  parallel  test  to  each  subject  (Samejima, 
1977c). 
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1 


TABLE  A-2-1 

The  Number  of  Hypothetical  Subjects  in  Each 
of  the  Success  and  Failure  Groups  of  Each 
Binary  Item  In  Degree  3 Case,  and  the  Nega- 
tive Number  (In  Parentheses)  to  be  Added  to 
Each  Frequency  to  Give  the  Corresponding 
Number  of  Subjects  In  Degree  4 Case 
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The  true  Item  characteristic  function  (curve) , and  the  frequency  ratio 
who  answered  the  Iten  correctly  to  the  total ^frequency  for  each  interv. 
2500  6 In  Degree  3 Case  (triangle),  2465  0 in  Degree  4 Case  (squari 

500  6 obtained  by  the  Normal  Approximation  Method  (circle). 


Item 


LATENT  TRAIT 

FIGURE  7-9 
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LATENT  TRAIT  0 

FIGURE  7-9 
(Continued) 

Exclusion:  Nornal  Approximation  Method,  both  the  500  6 ^and  2500 

Inclusion:  Normal  Approximation  Method,  the  other  2500  6 and  5000 
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Addition:  the  same  as  Items  2,  4 and  6(A) 
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The  number  of  Intervals  used  In  each  esClnaClon  Is  shown  In  parentheses. 


number  of  intervals  used  in  each  estlnatlon  is  shown  in  parentheses. 
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The  number  of  Intervals  used  In  estimation  Is  shown  as  a subscript  when  It  Is  less  than 
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niiinber  of  Intervals  used  In  esClnaClon  Is  shown  as  a subscript  when  It  Is  less  than 


